Skip to content

fix(lock): don't dereference a NULL locker in the priority accessors (#148) - #154

Merged
gburd merged 1 commit into
masterfrom
fix/lock-priority-nullderef
Sep 7, 2026
Merged

fix(lock): don't dereference a NULL locker in the priority accessors (#148)#154
gburd merged 1 commit into
masterfrom
fix/lock-priority-nullderef

Conversation

@gburd

@gburd gburd commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Fixes #148. Found by the coverage-driver work in #147, which SIGSEGVed while exercising the never-called getter surface.

The crash

DB_ENV->set_lk_priority() and DB_ENV->get_lk_priority() killed the process on a plain public-API call naming a locker id with no live locker — ~12 lines, no fault injection, no invalid arguments:

db_env_create(&e, 0);
e->open(e, path, DB_CREATE|DB_INIT_LOCK|DB_INIT_MPOOL|DB_INIT_TXN|DB_INIT_LOG, 0600);
e->set_lk_priority(e, 0, 100);   /* SIGSEGV */
Program received signal SIGSEGV
#0  __lock_set_lk_priority (priority=100) at ../src/lock/lock_method.c:483
#1  main () at r.c:13

__lock_getlocker() with create == 0 reports no such locker by returning 0 with a NULL locker, not by returning an error — its only assignment to *retp is lock_id.c:408, after the if (sh_locker == NULL && create) block. Both accessors tested only ret == 0 and dereferenced.

Why the check is local, not in the shared helper

__lock_vec_pp() (lock.c:93) passes the same possibly-NULL locker into __lock_vec(), which handles it deliberately:

"Since the locker may hold no locks (i.e., you could call abort before you've done any work), it's perfectly reasonable for there to be no locker; this is not an error."

So tightening __lock_getlocker()'s contract would break a caller that depends on today's behavior. The original report suspected __lock_vec_pp() was affected too — it is not, and I verified that rather than propagate it: lock_vec() with DB_LOCK_PUT_ALL and with DB_LOCK_UPGRADE_WRITE on an unused id both return 0 cleanly. Correction posted to #148.

Test

test/db/lock_priority_nullderef.c (5 checks) covers all three cases — EINVAL for setter and getter on an unused id, and a set/get round trip on a live locker so the fix can't regress into merely disabling the feature:

  set_lk_priority(unused id): ret=22 (Invalid argument)
  get_lk_priority(unused id): ret=22 (Invalid argument)
  live locker 2147483649: set/get round trip prio=77
lock_priority_nullderef: 5 checks, 0 failures

Proven to have teeth: reverting the two checks makes the runner report FAIL (rc=139, Segmentation fault). Wired into the CI regression-runner step added in #153.

Regression

5/5 test/db runners, test/lockmatrix, test/isolation, test/soak all PASS. (lock00*/txn00* TCL need --enable-test, which this local build lacks — covered by CI.)

Fixes #148.

DB_ENV->set_lk_priority() and DB_ENV->get_lk_priority() crashed the library on a
plain public-API call naming a locker id with no live locker:

  db_env_create(&e, 0);
  e->open(e, path, DB_CREATE|DB_INIT_LOCK|DB_INIT_MPOOL|DB_INIT_TXN|DB_INIT_LOG, 0600);
  e->set_lk_priority(e, 0, 100);   /* SIGSEGV */

__lock_getlocker() with create == 0 reports 'no such locker' by returning 0 with
a NULL locker rather than by returning an error -- its only assignment to *retp
is at lock_id.c:408, after the 'if (sh_locker == NULL && create)' block. Both
accessors tested only 'ret == 0' and then dereferenced. gdb:

  Program received signal SIGSEGV
  #0  __lock_set_lk_priority (priority=100) at ../src/lock/lock_method.c:483
  #1  main () at r.c:13

Both now return EINVAL for a missing locker. The check is deliberately local to
these two functions rather than a change to __lock_getlocker()'s contract:
__lock_vec_pp() passes the same possibly-NULL locker into __lock_vec(), which
handles it on purpose ('it's perfectly reasonable for there to be no locker;
this is not an error'), so tightening the shared helper would break a caller that
depends on the current behavior. The original report suspected __lock_vec_pp()
was affected too; it is not, and I verified that by calling lock_vec() with
DB_LOCK_PUT_ALL and DB_LOCK_UPGRADE_WRITE on an unused id -- both return 0.

Regression test test/db/lock_priority_nullderef.c covers all three cases: EINVAL
for the setter and getter on an unused id, and a set/get round trip on a LIVE
locker so the fix cannot regress into simply disabling the feature. Proven to
have teeth: reverting the two checks makes the runner report FAIL (rc=139,
Segmentation fault). Wired into the CI regression-runner step.

Found by the coverage-driver work in #147, which SIGSEGVed on the never-called
getter surface.
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

Coccinelle convention checks

No new violations. ✅

Resolved since baseline (2) -- update dist/cocci/baseline.txt to lock these in.
rule_mutex_unbalanced|MUTEX_UNBALANCED|src/crypto/mersenne/mt19937db.c|return (ret);
rule_mutex_unbalanced|MUTEX_UNBALANCED|src/mp/mp_register.c|return (ret);

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

ABI diff vs v5.3.34 (libabigail — authoritative)

Functions changes summary: 0 Removed, 0 Changed, 2 Added functions
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable

2 Added functions:

  [A] 'function int __lock_sireap_lockers(ENV*)'    {__lock_sireap_lockers}
  [A] 'function int __os_csprng(ENV*, void*, size_t)'    {__os_csprng}

Removed exported symbols (nm -D, _NNNN version suffix normalized)

None.


Advisory: libabigail/nm is the authoritative binary-ABI check; Coccinelle is complementary source-level early warning. See dist/cocci/README.md.

@gburd
gburd merged commit d63e2a5 into master Sep 7, 2026
53 of 55 checks passed
@gburd
gburd deleted the fix/lock-priority-nullderef branch September 7, 2026 04:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NULL dereference in __lock_set_lk_priority / __lock_get_lk_priority for an unknown locker id

1 participant